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Abstract Body 

Limit 4 pages single-spaced. 

Background / Context: Description of prior research and its intellectual context. 

Mathematics is the study of structure but students think of math as solving problems according to 
rules. Given a word problem, students might launch into calculating the correct answer - but 
mathematicians, given the same problem, may consider finding out classes of similar problems 
or representing such problems in an efficient and principled way. Students’ perception that math 
is about computing actually reflects a regularity present throughout their math experiences; they 
receive much instruction and practice on solving problems with immediately presented rules 
(Schoenfeld, 1989; Stigler & Hiebert, 2009). To solve, however, students must (at least 
implicitly) understand structure (Kellman & Massey, 2013). Repetitive practice with solving 
structurally identical problems reduces the need to understand structure. 

Students can learn procedures, but they often have trouble knowing when to apply learned 
procedures, especially to problems unlike those they trained with (e.g., Gick and Holyoak, 1983). 
Here we rely on the psychological mechanism of perceptual learning (PL), our natural ability to 
extract invariant information across multiple learning experiences (Gibson, 1969; Kellman & 
Garrigan, 2009), to bring about this early appreciation of structure. 

In two studies, we examined the effectiveness of a PL approach to word problems, and to explore 
the relationship between understanding structure and solving. We designed and tested a PL- 
based intervention called Math Problem Insight (MPI). In MPI, elementary school students 
practice solving and representing the structure of many word problems that varied along 
contextual and mathematical dimensions. The goal of MPI is to encourage the learner’s own 
attentional system to highlight and extract invariant structure from varied contexts, by providing 
variations of verbal structures that map onto related abstract representations. 

Purpose / Objective / Research Question / Focus of Study: Description of the focus of the research. 
The main research questions were: (1) Can PL interventions in which representation is the goal 
(rather than computing) help students make better generalization and use symbols to represent 
situations? (2) Are students who are proficient at solving are equally proficient in structure 
representation? 

In Study 1, we measured students’ improvements in solving and representing the structure of 
word problems from solving and mapping practice with a PL intervention The study also 
examined whether students were equally adept at solving and representing equivalent problems. 

In Study 2, we randomly assigned students to have only either representing experience (the 
Mapping condition) or solving experience (the Solving condition) during the PL intervention, to 
examine whether these particular short exercises led to task specific or general improvements 
with word problems. 


Setting: Description of the research location. 

In Study 1 , third-grade homeschoolers enrolled in an online school in Idaho completed the study 
from home, as part of typical online math assignments. Each student was individually assigned 
login names and passwords so that they could complete the assignment at their own pace. 
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In Study 2, third-graders from an elementary school in a Midwestern town logged in from their 
school computers to complete the study. This school expressed interest in the MPI intervention 
because they had a decrease in the percentage of students passing mathematics (as measured by 
their state’s standardized test) from 2011 to 2012. The teachers varied in how much class time 
was reserved for MPI. Most of the participants who completed the study (finished pretest, MPI 
learning trials, and posttest) came from a single class (18/21). The remaining three classes had 
fewer students complete the experiment (5/26, 2/26, 1/13). 

Population / Participants / Subjects: 

Description of the participants in the study: who, how many, key features, or characteristics. 

In Study 1,61 third-graders participated (31 in the experimental group, 30 in the comparison 
group. In Study 2, we included 26 third-graders who completed all portions of the study (10 in 
the Mapping condition, 16 in the Solving condition). 

Intervention / Program / Practice: 

Description of the intervention, program, or practice, including details of administration and duration. 

The Math Problem Insight (MPI) was designed to promote PL of one-step word problems that 
can be solved with addition, subtraction, multiplication, or division of integers. Consistent with 
the principles of PL, MPI consists of many brief interactive trials incorporating systematic 
variations. Students were asked to respond to these trials in one of two ways: (1) by solving the 
presented word problem (Solve trials; Figure 1A shows an example) or (2) by choosing the 
symbolic expression or short sentence that correctly represented the situation out of four 
presented options (Map trials; Figure 2A shows an example). In Study 1, the MPI consisted of 
both Solve and Map trials. In Study 2, students were randomly assigned to a special version of 
the MPI that included only one type of learning trial (Solve or Map). 

In MPI, there were 14 categories of addition/sub traction (AS) and multiplication/division (MD) 
word problems represented in the database of almost 2500 problems, defined by their isomorphic 
structures using the classification system developed by the Cognitively Guided Instruction (CGI) 
group (Carpenter et ah, 1999). Within each AS and MD category, word problems were 
constructed from a variety of contexts (Please see Tables 1 & 2 for examples). 

There was no explicit instruction. Instead, variant and invariant information were present across 
the trials. For a given mathematical relationship, multiple classification trials revealed structural 
invariance in changing contexts. Across different types of trials, students had to interact with 
different structures, allowing contrasts and relations to emerge. Incorrectly answered trials (of all 
types) were followed with animated feedback showing a horizontal bar visualization that 
modeled the situation (shown in Figures IB and 2B). Corrective feedback was designed to 
highlight structural components in a similar way across trials. 

Problems were presented in an mixed fashion using the ARTS adaptive learning system (Mettler, 
Massey & Kelhnan, 2011), in which accuracy and speed of response to instances of each 
category were used to space and sequence subsequent learning trials uniquely for each learner. 
ARTS also implements objective learning criteria involving accuracy, speed, and maintenance of 
proficiency across spacing intervals. Students finished the MPI learning trials when they 
correctly answer at least 4 out of 6 trials of a particular category and in roughly 30 seconds per 
trial (speed criteria varied depending on the text length in the word problem). 
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Research Design: Description of the research design. 

Study 1 : pretest- posttest design with a matched comparison group. The comparison group 
received conventional mathematics lessons that overlapped with the MPI content in tenns of 
using basic mathematical operations in problem solving, and took the online assessment only at 
the time of posttest. 

Study 2: random assignment experimental design. One group participated in the MPI with only 
Solve trials. The other group participated in the MPI with only Map trials. 


Data Collection and Analysis: Description of the methods for collecting and analyzing data. 

In both studies, the pretest and posttest were a representative sampling of problems from all 
categories, incorporated seamlessly as part of the MPI experience. The pretest was administered 
prior to learning trials, and the posttest was presented after the participant retired their last 
category. No feedback was given on pretest and posttest trials. The trials on the pretest, posttest, 
and MPI consisted of Solve, Unknown-Left Map (when the unknowns are depicted on the left of 
the equal sign), and Unknown-Right Map problems (when the unknowns are on the right of the 
equal sign). All posttest problems were novel instances of the structures experienced during MPI. 
We recorded and analyzed the accuracy and response time of each trial in both assessments and 
during the MPI. 

Study 1: (1) Can we train students to accurately solve and represent word problems using 
principles of PL? We compared pretest and posttest performance of the experimental group using 

2 (Test: pretest, posttest) x 3 (Trial-type: Solve, Unknown-Left Map, Unknown-Right Map) 
repeated measures ANOVAs on accuracy and response times, and compared posttest 
performance between the comparison and the experimental groups using 2 (Condition: Solving, 
Mapping) x 3 (Trial-type) mixed ANOVAs. (2) To examine whether students’ solving 
proficiency relates to their ability to represent mathematical structure, we compared their 
accuracies in Solve trials and in the two types of Map trials during the MPI learning trials with a 

3 (Trial-type) x 2 (Operations: AS, MD) repeated-measures ANOVA on accuracy. 

Study 2: Because of the limited time allotted for MPI during class time, our data may have 
included a greater proportion of highly proficient students who were able to complete the MPI. 
Because of this possibility, we included performance on the MPI learning trials (proportion of 
correct trials) as a covariate. To determine which version of MPI improved students’ 
performance, accuracy was examined with a 2 (Condition) x 2 (Test) x 3 (Trial-type) mixed 
repeated measures ANCOVA. Condition was a between-subjects variable while Test and Trial- 
type were within-subjects variables. 

Findings / Results: Description of the main findings with specific details. 

Table 3 shows the results of Study 1, and Table 4 shows the results of Study 2. 

Study 1: (1) Mapping of word problem to its algebraic representation is very difficult for 
students, but after MPI training, students improved in their ability to solve and represent word 
problems, especially more so in Solve trials than in both Unknown-Left and Unknown-Right 
Map trials, all t( 15) > 3.79 , p < .003, d > .97. There were also large changes in fluency from 
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MPI practice, as indicated by changes in RT of correct responses in all trials, t(30) = 4.37, p < 
.001, d = .79. Students initially took an average of 33 seconds per problem. After MPI, the 
average RT reduced to 20 seconds, a 38% drop in solution time. Compared to the comparison 
group, the MPI showed limited benefit of accuracy, but a resounding improvement in fluency 
with word problems, all t( 59) > 4.81,/? < .001, d > 1.23. On average, the MPI group only needed 
19 seconds per problem while the comparison group spent 49 seconds on the same problems. (2) 
In the MPI learning trials, students were significantly more accurate in Unknown-Right than 
Unknown-Left Map trials, F(l, 30) = 14.62, p < .002, rf = .33; also, they were more accurate on 
Solve trials than both types of Map trials, F( \ , 30) > 28.27, p < .001, rj 2 > .49. It is not surprising 
that students generally struggle on Unknown-Left Map trials, but the difference between Solve 
and Unknown-Right Map trials is surprising: students can identify and compute an operation 
more easily than simply identifying it. 

Study 2: Both solving and mapping activities contributed to learning. In particular, the Mapping 
condition improved in the Solve trials, t( 9) = 2.28, p = .049, d= .82, and in Unknown-Right Map 
trials, t( 9) = 2.93 , p = .017, d = .93. The Solving condition also demonstrated improvements, 
namely on Unknown-Left Map trials, t(\ 5) = 2.7 5, p = .015, d= .71. Solving practice contributed 
to improved mapping performance and mapping performance also fostered better solving. 


Conclusions: Description of conclusions, recommendations, and limitations based on findings. 

For elementary math students, identifying the operation for a word problem is a distinctly 
different task than computing the solution to the word problem. Training that emphasizes 
perceptual learning through mapping across multiple isomorphic representations improves 
students' pick up of abstract structure. Additionally, PL training with a particular task generalizes 
to other tasks that involve related mathematical structures. This replicates findings of other PL 
interventions in mathematics (e.g., Kellman et ah, 2008; Kellman, Massey, & Son, 2009). Our 
results suggest a subtle advantage for mapping activities over simply solving. This is notable 
because mapping across representations are uncommon in elementary math pedagogy. An 
important difference between the two studies was that the first demonstrated general fluency 
gains from pretest to posttest while Study 2 did not (instead there was evidence of learning gains 
for specific types of trials). This may be due to the oversampling of highly proficient students 
(pretest RTs in Study 2 were similar to posttest RTs in Study 1). Additionally, these differences 
may be attributable to the characteristics of the population sampled in the two studies. These 
differences raise questions about the potential generalizability of online learning interventions 
(with minimal teacher/parent scrutiny) for a variety of populations. 

Finally, although Studies 1 and 2 are difficult to compare directly due to different participant 
groups, the results were consistent with the possibility that the strongest PLM intervention 
effects occurred when mapping problems, requiring interactions with problem structures across 
representational formats, were mixed with solving practice also specifically designed to enhance 
PL. This result may have important implications for instructional interventions and deserves 
further investigation. 
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Appendix B. Tables and Figures 

Not included in page count. 


Incorrect 

Stephanie has 2 cents. Bryce also has some cents. 
Altogether, they have 10 cents. How many cents 
does Bryce have? 


12 


The correct answer was: 8 



10-2 = 8 
cents all together Stephanie’s cents Bryce’s cents 


Figure 1. (A) Screenshot of a Solve trial showing a blank for the solution of the given word 
problem. (B) The animated feedback shown if the chosen option is incorrect. 


Incorrect 

c \ 

Alexis places 5 marbles in each bag. How many 
bags does Alexis need for 15 marbles? 



B ^_A /v " 

5x3 = 15 

marbles in bags marbles 

each bag 

Figure 2. (A) Screenshot of a Map trial showing four possible options. (B) The animated 
feedback shown if the chosen option is incorrect. 


Note: To help students make connections across problems, the structural components were 
consistently depicted. These structural components are given context specific labels relevant to 
each problem (i.e., instead of units per group, students would see marbles per bag). For instance, 
in the MD feedback animations (see Figure 2B), units were shown as small green squares; 
groups were shown as simple gray brackets that group the squares, and units per group were 
shown as a highlighted group of green squares in the left-most bracketed group. In the AD 
feedback animations (see Figure IB for an example), the start value was depicted as a yellow 
bar, the change value was depicted in green, and the result was depicted in red. 


SREE Spring 2014 Conference Abstract Template 


B-l 



Table 1. Example AS problems based on CGI classifications (Carpenter et al., 1999) 



Start Unknown 

Change Unknown 

Result Unknown 


(subtraction) 

(subtraction) 

(addition) 


Jesus has animal 

Karen has 8 crayons. 



crackers. Brittney gave 

Darin added some more 

Evan has 8 stickers and 

Join 

him 5 more. Now he 

crayons. Now Karen 

gets 5 new stickers. 


has 13 animal crackers. 

has 13 crayons. How 

How many stickers 


How many did Jesus 

many crayons did Darin 

does Evan have now? 


have to start with? 

add? 


Separate 

Alvin has 13 dollars 
and gives away 5 
dollars. How many 
dollars does Alvin have 
now? 

Nina had 13 brownies 
and gave away some of 
them. Now, she has 8 
brownies. How many 
brownies did Nina give 
away? 

Monica gives away 5 
cents and now has 8 
cents left. How many 
cents did Monica start 
with? 

Part-Part- 

Whole 


[Part unknown] 
Wyatt had 8 walnuts 
and some hazelnuts. In 
total, he has 13 nuts. 
How many hazelnuts 
does Wyatt have? 

[Whole unknown] 
Ming has 8 sculpture 
and 5 paintings. How 
many art pieces does 
Ming have total? 

Compare 

[Quantity 1] 
Olivia became 5 weeks 
older. Olivia is now 13 
weeks old. How old, in 
weeks, was Olivia 

[Difference] 
Leslie has 8 roses. 
Arianna has 13 roses. 
How many more roses 
does Leslie have than 

[Quantity 2] 
Michael walked 8 
kilometers. Chloe walked 
5 more kilometers than 
Michael. How far, in 
kilometers, did Chloe 
walk? 


before? 

Arianna? 
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Table 2. MD example problems from four contexts based on CGI classifications (Carpenter et 
al, 1999) 



Units per Group 
(partitive division) 

Groups 

(measurement division) 

Units 

( in ultiplication) 


Jasmine places 24 


Wei observes 4 

Objects/ 

Groups 

marbles in 6 bags, with 
the same number in 
each. How many 
marbles are in each 

Nicole puts 4 apples in 
each barrel. How many 
barrels does Nicole 
need for 24 apples? 

sculptures in each 
museum. How many 
total sculptures does 
Wei observe in 6 


bag? 


museums? 


Giovanni bikes 24 

Emily travels 4 miles 

Taylor drives 4 

Distance/ 

Time 

yards in 6 seconds. 
How many yards per 
second did Giovanni 

per hour. How many 
hours will it take for 
Emily to travel 24 

kilometers per week. 
How many kilometers 
does Taylor drive in 6 


bike? 

miles? 

weeks? 

Money 

(Exchange) 

Joseph buys 6 pies for 
24 dollars. If each pie 
cost the same amount, 
how much does one 
pie cost? 

Logan earns 4 dollars 
per hour. How many 
hours does Logan need 
to earn 24 dollars? 

Maya trades 4 coupons 
per doll. How many 
coupons does Maya 
trade for 6 dolls? 

Compare 

[Quantity 1 missing] 
Misaki's plant is 4 
times as tall as Holly's 
plant, which is 6 feet 
tall. How tall, in feet, 
is Misaki's plant? 

[Scalar multiplier] 
Luke's fence is 6 meters 
tall. Martina's fence is 
24 meters tall. Martina's 
fence is how many 
times as tall as Luke's 
fence? 

[Quantity 2] 
Sergey's beard is 6 
inches long, and is 4 
times as long as 
Howard’s beard. How 
long is Howard’s beard? 
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Table 3. Study 1 ’s mean proportion correct and RTs of correct responses on pre- and posttest 
me asures. Standard d eviati ons are shown in parenth eses. 



ACCURACY 

RESPONSE TIME 




(in seconds) 


Pretest 

Posttest 

Pretest 

Posttest 

ALL STUDENTS 





MPI GROUP (n = 31) 





Solve 

.81 (.21) 

.89 (.13) 

29.7 (14.4) 

18.3 (13.5) 

Map, Unknown Left 

.60 (.28) 

.77 (.23) 

34.3 (19.3) 

20.6 (14.0) 

Map, Unknown 
Right 

.67 (.24) 

.76 (.22) 

33.5(16.1) 

20.0 (12.2) 

COMPARISON GROUP 





(n = 30) 





Solve 

— 

.86 (.17) 

— 

44.6 (25.5) 

Map, Unknown Left 

— 

.62 (.26) 

— 

61.8 (45.6) 

Map, Unknown 
Right 


.73 (.25) 


46.7 (26.3) 

PRETEST ACCURACY < .80 





MPI GROUP (n = 16) 





Solve 

.69 (.21) 

.87 (.15) 

32.4(15.4) 

22.0(17.8) 

Map, Unknown Left 

.43 (.25) 

.76 (.23) 

33.5 (23.6) 

24.8(18.1) 

Map, Unknown 
Right 

.53 (.23) 

.70 (.25) 

38.9(18.0) 

22.5 (16.0) 

COMPARISON GROUP 





(n = 14) 





Solve 

— 

.75 (.19) 

— 

47.6 (23.3) 

Map, Unknown Left 

— 

.47 (.25) 

— 

62.9 (60.8) 

Map, Unknown 
Right 

— 

.56 (.25) 

— 

39.4 (20.9) 
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Table 4. Study 2’s mean proportion correct and RTs on pre- and posttest measures. Standard 
deviations are shown in parentheses. 


RESPONSE TIME 

ACCURACY (sec) 



Pretest 

Posttest 

Pretest 

Posttest 

SOLVING CONDITION (n = 

16) 




Solve 

.80 (.14) 

.83 (.17) 

17.9 (7.9) 

15.9 

(11.2) 

Map, Unknown Left 

.53 (.27) 

.70 (.19) 

22.1 (10.0) 

21.7 (9.5) 

Map, Unknown Right 

.66 (.20) 

.67 (.26) 

19.7 (6.9) 

21.1 

(11.8) 

MAPPING CONDITION (n = 

10) 




Solve 

.73 (.18) 

.83 (.26) 

15.9(11.2) 

19.8 (6.7) 

Map, Unknown Left 

.61 (.28) 

.66 (.24) 

21.7 (9.5) 

17.1 (7.4) 

Map, Unknown Right 

.63 (.21) 

.81 (.22) 

21.1 (11.8) 

19.6 

(13.0) 


SREE Spring 2014 Conference Abstract Template 


B-5 



